# Benchmark Dataset

The TOFU dataset contains question-answer pairs generated based on 200 fictional authors that do not exist. It is used to evaluate the forgetting performance of large language models on real-world tasks. The task aims to forget models fine-tuned on various forgetting set ratios. This dataset uses the question-answer format, making it suitable for popular chatbot models like Llama2, Mistral, or Qwen. However, it can also be used for any other large language model. The corresponding codebase is written for Llama2 chatbot and Phi-1.5 models but can be easily adapted to other models.

Emu Edit is a multi-task image editing model that performs precise image editing by recognizing and generating tasks. It has made the latest technological breakthroughs in this field. Emu Edit's architecture is optimized for multi-task learning and trained on numerous tasks, including region-based editing, free-form editing, and computer vision tasks such as detection and segmentation. In addition, to more effectively handle these various tasks, we have introduced the concept of learned task embeddings to guide the generation process for accurately executing editing instructions. Our model, through multi-task training and the use of learned task embeddings, can significantly improve its ability to accurately execute editing instructions. Emu Edit also supports rapid adaptation to unseen tasks through task inversion for few-shot learning. In this process, we keep the model weights unchanged and only update the task embeddings to adapt to new tasks. Our experiments demonstrate that Emu Edit can quickly adapt to new tasks such as super-resolution and contour detection. This makes Emu Edit particularly advantageous for task inversion when labeled samples are limited or computational budgets are restricted. To support the strict and well-founded evaluation of instruction-based image editing models, we have also collected and publicly released a new benchmark dataset containing seven different image editing tasks: background modification, global image change, style modification, object removal, object addition, local modification, and color/texture modification. In addition, to allow for a fair comparison with Emu Edit, we also share Emu Edit's generation results on the dataset. Emu Edit 2023 Meta retains all copyrights

Awesome-Domain-LLM is a project that collects and organizes open-source models, datasets, and benchmark datasets for vertical domains. The project includes open-source models, datasets, and benchmark datasets for multiple domains, including medical, legal, finance, and education, aiming to promote the empowerment of various industries by large models. Users can find models and datasets suitable for their fields in this project to improve work efficiency and quality.